Interpopulational Nucleotide Differences

A mathematical theory is developed for computing the probability that m genes sampled from one population (species) and n genes sampled from another are derived from 1 genes that existed at the time of population splitting. The expected time of divergence between the two most closely related genes sampled from two different populations and the time of divergence (coalescence) of all genes sampled are studied by using this theory. it is shown that the time of divergence between the two most closely related genes can be used as an approximate estimate of the time of population splitting ( T ) only when T = t l ( 2 N ) is small, where t and N are the number of generations and the effective population size, respectively. The variance of Nei and Li's estimate ( d ) of the number of net nucleotide differences between two populations is also studied. It is shown that the standard error ( s d ) of d is larger than the mean when T is small ( T << 1). In this case, sd is reduced considerably by increasing sample size. When T is large (T > I) , however, a large proportion of the variance of d is caused by stochastic factors, and increase in the sample size does not help to reduce sd. To reduce the stochastic variance of d , one must use data from many independent unlinked gene loci. FTER the discovery of the molecular clock (ZUCKERKANDL and PAULING A 1965), many authors have attempted to estimate the time of divergence between species or populations from amino acid or nucleotide sequence data. Although the molecular clock does not run as regularly as the ordinary clock, it gives a rough idea about the divergence time (FITCH 1976). When this method of estimating divergence time is applied to closely related species or populations, however, some caution is necessary because the divergence time between a pair of genes (or proteins) sampled from different populations may be substantially greater than the time of population splitting (Figure 1). This situation occurs when the ancestral population is polymorphic. When more than two genes are sampled from each population, a correction for the effect of ancestral polymorphism can be made, and the number of nucleotide substitutions (net nucleotide substitutions), d , that occurred after population splitting is estimated by subtracting within-population differences (NEI and LI 1979). ' Present address: National Institute of Genetics, Mishima, Shizouka-ken, 41 I Japan Genetics 110: 325-344 June, 1985. 326 N. TAKAHATA AND M. NE1 a b c d e f FIGURE 1 .-Genealogy of six sampled genes, three (a, b and c) from one population and three (d, e and 9 from the other population. The two populations are assumed to have diverged at time T . T I , Ts and Ts represent the times of gene splitting. When nucleotide substitution occurs by mutation and genetic drift, the expectation of d is given by E ( d ) = 2vt, where v is the mutation rate per gene and t is the time since divergence between populations X and Y (LI 1977; NEI and Lr 1979). Therefore, if we know v , we can estimate t from d . However, to evaluate the accuracy of this method, we must determine the variance of d generated by both sampling and stochastic errors. The sampling variance was studied by NEI and TAJIMA (1981)) but the variance due to stochastic errors has not been worked out. To evaluate the variance of d generated by both sampling and stochastic errors, we must first know the expected genealogy of sampled genes. Knowledge of the expected genealogy of sampled genes is also important for estimating the difference between the time of population splitting and the time of gene splitting. If this difference is small, the time of population splitting may be estimated approximately by the time of gene splitting. The main purpose of this paper is to study the above two problems. We first examine the relationship between population and gene splitting times and then investigate the variance of d. Throughout the paper, we assume that one population splits into two (populations X and Y) t generations ago and, thereafter, no migration occurs between them. We also assume that the effective population size (N) remains constant throughout the evolutionary process, and, thus, the populations are in steady state with respect to the effects of mutation and genetic drift. We use the infinite-site model of neutral mutations with no recombination (KIMURA 197 1 ; WATTERSON 1975). GENEALOGY OF GENES SAMPLED FROM TWO POPULATIONS We first consider the expected genealogy of m genes sampled from population X (or Y) and derive a formula for the probability that the m genes are GENES IN RELATED SPECIES 327 descended from mo at the time of population splitting. Obviously, 1 5 mo 5 m. We start with the formula for the probability density vp-l(s)] of waiting time s at which p genes are descended from p 1 genes. We use the continuous time approximation to the Wright-Fisher model and apply KINGMAN'S (1 982) equation where ap = p ( p 1)/2 (see also HUDSON 1983; TAJIMA 1983; TAVAR~ 1984). Equation 1 may be written as fp-1(7) = ape-*P', (2) if we measure time in terms of T = s/(2N). To obtain the probability distribution of the number of ancestral genes (mo) at the time of population splitting [T = t/(2N) units of time ago], we introduce a set of random variables ( T ~ ) . ~ ~ 1 is the waiting time at which p genes are descended from p 1 ancestral genes and follows the exponential density function given by (2). For 1 5 mo 5 m 1, we define the sum and denote by p(Smmo = T ) the probability density of S,,, = T . Equation (3) represents the waiting time at which m genes are descended from mo ancestral genes. p ( S m m , = T ) can be obtained by the convolution of fmo, , . . , f m z , f m l . Using the Laplace transform and the partial fraction expansion, we obtain where m Pp(m, mo) = n (a, a$'. r=mo+l +P ( 5 ) The probability, Pmmo(T), that m genes are descended from mo ancestral genes T units of time ago can be obtained by using (4). That is, P m m ( T ) = p b ' m m o T 5 Smm, + T m o l ) (6) T m = d T P ( S m m o = 71 dr~moeXP(-&oo,